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ABSTRACT 

Recently, educational evaluation has attempted to use 
the precision, objectivity, and mathematical rigor of the 
psychological measurement field as well as to find ways in which 
instrumentation and data utilization could more directly be related 
to educational institutions, educational processes, and educational 
purposes- The linkages between educational purposes and educational 
e'valuation are so strong that systematic workers in most areas of 
educational research or practice start with this almost as the first 
step in their work. The actual materi^ils of instruction and the 
observation of teaching-learning situations can be analyzed to 
determine the appropriate evaluation procedures, and, in turn, the 
relations between the stated objectives, the learning experiences 
available to students, and the evaluation procedures can be 
determined in great detail. Evaluation instruments serve as models 
for teaching and learning and, as such, help to guide both 
instruction and student learning. Mastery learning is one way in^ 
which luation can be integrated in teaching/learning. Evaluation 
techn: i: are also available to measure the effectiveness of 
interav ion and learning in innovative and alternative programs. 
Increasingly, Educational evaluation is seen as a quality control 
measure. (Author/IRT) 
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EVALUATION, INSTRUCTION, AND POLICY MAKING 

By 

, ^ C. S. Bloom 

Professor of Education 
University of Chicago 

♦B ackground 

Although educational measurement has existed in some form or other for 
several thousand years, much of its development into a complex art and technology 
has taken place during the 20th Century. During much of this century the field 
has been dominated by the ideas of psychologists, psychometricians , and 
statisticians. It is only "within the past few decades that educational evalua- 
tion has sought to free itself from these ancillary fields in order to find 
clearer roots in the educational process and educational concerns and problems. 

Psychological and educational measurement was primarily concerned with 
the development and utilization of instruments that could be used for prediction, 
selection, and certification in ^relation to students and student achievement. 
Such functions could be served by specialists far removed from education and 
educational processes in the schools. And, in fact, most of the educational 
Measurement specialists were trained in psychology and statistics with little 
grounding in the field of education or even educational psychology. 

The more recent field of educational evaluation has attempted to make 
use of the precision, objectivity, and mathematical rigor of the psychological 
measurement field, but in addition has sought to find ways in which 
instrumentation and data utilization could more directly be related to 
educational institutions, educational processes, and educational purposes, 
In this paper, I will attempt to sketch some of the major dimensions of this 
work as they appear at this time, I am confident that this field will 
develop in many new ways and that we can only dimxy perceive a few of the 
niajor lines this work will take. in the future. 



- 2 . 



Educational purpose and educational ovaluation 

Educational purposes, goals, and objectives have been with us yince- 
the beginning of i*ormal education • ^ Expressed in verbal form these statements 
of intentions were useful in giving a general direction to the educational 
institution ^but only rarely were they operational statements which guided * 
either the teacher or the learner. 

In sharp contrast, the instruments for educational measurement (external 
examinations, teachers* tests and final examinations, otandardized tests, ^tc.) 
have always had a controlling force on what was taught and, even more, on 
what was learned by students. Since the major rewards and penalties of an 
educational system are tlBd to its certification and grading procedures, 
which in turn are dependent on its examination procedures, the teaching- 
learning activities of teachers and students are to a large extent ^juided 
by what they expect will be tested on these examinations. And, in 
countries throughout the world, the examination procedures have been largely 
limited to a single objective — the testing of recall of specific information 
about each school subject. 

Perhaps the major innovation of educational evaluation was the 
development of ways in which the evaluation process could be integrally 
related to the educational purposes of the Classroom, the school, and 
the national educational system. Much progress in this work has been 

doctimented in the many book.i on educational evaluation, taxonomies of 

■3 . ■ • 

educational objectives, ahd curriculum evaluation. While there are many 
differing views about how the objectives should be defined, who has 
responsibility for determining the objectives, and the precise procedures 
for evaluating each objective, there is much concensus throughout the world 
on the importance of relating educational evaluation to educational purposes. 
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Starting with the pioneering work of Ralph Tyler in the 1930*8, the 
development of evaluation procedures for specific typea of educational 
objectives has moved with careful research and experimentat^ion until it 
haa reached the stage of what mi^ht be termed a technoXojy* While there 
are still many opportunities for creativity and artistry in the construction 
and US6 of evaluation procedures, the models and techniques for developing 
evaluation procedures for major classes af cognitive and affective objectives 
have been op*icified in relatively clear detail. Having been involved in 
this work for over three decades, I have been surprised an ) delighted to 
find that noat of ray students can develop the neceasary skills for this 
work in 3 to 6 months in contract to the several years necessary to develop 
similar skills in the 1940's. I- attribute much of zhis to the fact that 

9 

the procedures are now more clearly developed and illustrated in the many 
books and manuals on educational evaluation. 

It is common practice now for all the major educational testing 
organizations to start the construction of a new educational test with 
a detailed set of specifications of the conttmt and objectives to be 
tested and then to check the validity of the test items against the 
detailed specifications. Similarly, every new ourrirulum or research 
project or evciluation program starts with the specifications to be met 
in terms of content and objectives, then develops instruments, sampling 
procedures, research desigr*, ar.d data analysis in terms of these 
'specifications. The point is that the linkages between educational 
purposes and educational evaluation are so strong, th.it systematic 
workers In most areas of educational research or practice start with 
this almost as the first step in thiir work. Also., ehe detailed 
procedures for making the link^ages are so well developed that evaluation 
workers can be trained to do it well in much less time than was true 
several decades earlier. ^7 



Bducational evaluation aa models for teachinqi and learning; 

One of th« consequences of the linkage between educational purpose 
and educational evaluation is that the evaluation procedures btfcome operational 
definitions of educational purposes. It is now possible to clasL-ify the 
Items » problems, and procedures being used in examinations, tests, 
questionnaires, observational forms, and other evaluation material 
and techniques to determine what purposes are being respreaented by 

the evaluation techniques. Thus, as in the IKA studies, a collection 

I 

of the evaluation procedures being used wilhin a nation. when properly 

analyzed ^ give more operational information about the educational objectives 

of a subject of study or curriculum of a school or country than does the ^ 

verbal statements about the course or curriculum (Bloom, 197,4) ."^ 

Furthermore, the actual mater'. Is of instruction and the observations 

of teaching-learning situations can be analyzed to determine the appropriate 

evaluation procedures and in turn the relation between the stated object! /ea, 

the learning experiences available to student., and the evaluation 

procedures can be (determined ^in great detail. From the.se analyses, one 

can get a better picture of the kinds of learning being developed in a 

classroom, school, or entire educational system than is likely to be 

true from observations which might take several years to carry out . Theae 

analyses are very effective in predicting (and accounting for) the kinda 

of learning eventually found on major national or international survey 

2 3 

instruments such as those found in the I£A studies, Plowden Report, and 
4 

Coleman Keport. 

But, there are even more important consequences of the linl^^age 
between educational purpose and educational evaluation. One can 
determine where the linkeages are distorted between educational purposes, 
instruction, and evaluation. Is it that the purposes €u:e beyond the present 
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capabilities of the evaluators to develop appropriate uvaiuation procedure i? 
If 30, then tho task of training; educational evaluatori: to construct more 
valid and appropriate evaluation procoduroa. bocomea rlearl Is it that 
the teachers have not yet learned how to provi:ie inatructi^ for partic^ular 
educational purposes? If so, then the need for pre-serviv-e and in-service 
education of teachers beconen cloar. .If thp taak of provi.-iinf: nuch training 
appears inaunnnuntable for econoaic' reasons or because of tiio oreuent 
capabilities and training of the teaching r^taff, then .:an the situation be 
remedied by improvements in the instructional materials; by the use of 
radio, TV, or educational films; or by the une of peer tutoring and other 
special instructional proce'dures? 

It in evident that throughout the world students attempt to learn 
the skillr, abilities, and subject content that they believe will be 
emphasized' in the evaluation procedures they will be judged on. If they 
believe this is largely rote information, tiiey will study and prep/:re' 
accordingly. If they believe they will be jud(.:ed or. tl:eir ability to 
use the ideas and processes in new situations, they will learn and prepare 
to demonstrate such abilities. There has beL?n a great deal of observational 
studies as well as more direct experimental research on how students 
learn and preprire in relation to different kinds of examinations. The 
evidence is unmistakeable — students will attempt to learn what tliey 
anticipate will be emphasized in the evaluation instruments on which they 
expect to ue judged, graded, and certified. There is little doubt that 
a series of major changes in the evaluation procedure:^ over a number of 

o 

years can bring about great changes* in the learning of chc rtudents — 
probably more change than could be produced by any othor ingle change in 
the educational situation. ^ This is, of course, a two-eri^; 3d sword in that 
negative changes (reduction in the quality of learri»^'-) n^: well as positive 



changes (inprovomeota in th^^quality of learning) con be produced by 
related chanties in the evaluation proceduroa. But, thy p< int of this ^ " 
relation between otudent loarnng and '/alUGtion* is that the evaluA-tion 

> proceaurea furnish models of what learning is expectea and the uodels 
♦ are clearer than -the more ambiguous statements of educ u;,i.oiiai purposes 

-or the complex range of inatructional materials and proct^dures the 
students have been exposed to^ The clearest guide that students havq 
as -to what learning (lar^^ely cognitive) is expected oi* them is the 
evafuation instruments .on which they will bf< judged and ^=,Taded. 

Similarly, teachers are also guided by the evaluation procedures as 
to what they are to teach and what will* be expected of thoir students- 
Even when the evaluation procedures are made by ^ the to^cK^r himself, they 
define the end learning products of his. teaching 'and he strives to 'prepare 
Jiijf students to do well on these evaluation iBstrum^'^n^:'. If the ovaluatio' 
procedures largely deal with rote types of learning, toacheri3 will 
prepare their students .for such types of evaluation^' If tKo evaluation 
procedures largely deal with application of ideas to nev problems (euch 

■i 

as are exemplified in open-book examinations where ytuients may refer to 
their notos or books as they wish), then teachers will attempt to develop 
these kinds of learning in thoir students. It has been found that one 
of the most effective ways of prejparing teachers to teach higher mental . 
processes is to develop skills. for testing such proconbes in the teachers 
and to he .hem include problems of the appropriate tyi/e in their own 
evaluation procedures. 

Evaluation as an integral part of instruction and learain/? 

. Evaluation instruments do serve as models for teaching and learning 
and as such help to guide both the instruction as well as student l^aminga 
Eva luatioi^ used this way is largely a perceptual phenomena in that teache.rs 

10 



and students have expeotatione as to the evaluation procoiurea to be used 
(sometimes incnrrHctly) and ^:uide their r^fforts by th^ir anticipations. 
Tihus, the ovaliiation procciuren servft to indinate the /^oala to be reached 
.a«t' the end aome, period oT inatniction and ioHrning (ur.ually the end of 
the academic tenD)i • 

M^ny of (un. hhve oearched-for ways in which ovaluai .on mi^^ht bncomo n 
more integral p^rt/of the process oft teaching .and learnirv^ during the 



actual process Ve had become aw^are ol the effi»cts of the frequency oi 
testing on t'he -learning; of studt^ats (typic^illy thr Jiozc fre-^ucnt the tenting 
the hi^^her the achieveiAe#t)j the ways in which nome teachers analyze the 
resulj^s of tl^clr progress tests and quizzes to ieterciir.- :\erein they 
should stress certain poir:t8, review others, and oven provide 5ipf?cial hcln 
to students whQ have difficulties; and the effecjg of rhe kind of testing 
and the fh?<^uency of testing on t^e preparation th^t students make as well 
as the pacin(r of their learning activities. 

in*addit^ion, we became aware of the effects of group instruction on 
the differential If^arnihg cf students within a class c Much of th** research 
on clas3rccn ir^ struct ion. had demonr^trated that students differ in^their 
learning even though theore,ti r:ally all had equal opportunity to letTm in 
the some classroom.. Ve conceived, of this differential learning as errors 
in both instmictipn and 'learning and we took the position that if errors 

in student learning are systematically corrected at^each stage in the 

learning process there should be little variation in the final outcomes 

vas measured by' a summative evaluation measure. Purtherr^re, .'Students who 

have been corrected at each stage of the learning. shoMir^,..ichieVY? nt tnuch'*^ 

higher level than other students who have 'not been helped whjBri they needed 

I 

it *«even though both groups of students. were in the same classroom '^b^ were 
taught . similarly by the same teacher. ^ * 
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Thia ayatf»natic corroctivo learninK ha'J bf*en teraci mnntery leamin^r 
and thrre nre a number of teuching atrategies to achievt; i3urh ma!5tory« 
Central to noyt aaiitory loarnin^ strategies is the' uh*^ ol' feedback and 
correctivo procodurea at varioua ata^ea or parts of the loarnin^r pi'ooena. 
While a variety of feedback proceasea are possible (including q\ji:'.zet'i , 
homework, workhooka, etc.) it iiaa been fount! that the development' and use 
of brief dia^rnoatic teats haa proven to be moi^t effective. Such dia^iontlc 
or formative teats are* Intended to determine what each learner has learnod 
in a particular unit, chapter, or part of the course and what he still needa 
to learn. In general, these formative tests are not used to grade or jud^je 
the student and their main value is in providing' feedback to ooth teachers 
and studenta on wliat aspects or elements of t)ie learning unit still need 
to be mastered. The sucrreaa or failure of mastery learning work ia clearly 
related to, the degree of efficiency of these formative tenta in pinpointing 
the learning needs of each atudent. 

> 

However, the key to the success of mastery learning strategics largely 
lies in the extent to which students can be motivated and helped to correct 
th^ir learning difficulties at the appropriate points in the learning 
proceas. Many teachers have been very effective in motivating students -to 
do the necessary additional work and in finding ways of providing the 
correctives they need. The research done, so far in the United Stiitea, 

Canada, S. Korea, and a number of other countries suggests that the 
development of a ^student partner system or providing opportunities for 
groups of 2 or 3 students to. work together are vei*y effec'tive methods 
of motivating each student to make the'^correcti\''es and in addition thia 
provides \he additional time and help each atudent nec:^s. Teather aides, 
pro^fammed instruction,- audio tapes or cassettes, and other instructional 
material appear to work well in particular situations. In moat. cases 



throughout the world, the corrective work following the formative test 
■feedback io done outside of the regular clasaroom time. 

In the man^r studies reported by Block ( 1971, 1974) ^and by Peterson (l97?) 
there is considerable evidence that mastery learning proo'^dures do work well 
in enabling about of students to reach a level 01' acnit^vonent which 
less them PO?^ of students attain laider non-mastery or conventional teaching 
methods. The time costn for the mastery learning is typi>.£illy of the order 
of 10^ to 20ji additional time over the classroom scheduled time — for those 
students who need it. In a number of studies, it haa c^en found that the 
extra time and help needed decreases until tov/ard the end of the course 
little or no correctives are needed to attain the criterion of mastery 
on the formative tests. (Bloom, 1973)*^ 

While there are many different approaches to the improvement of both 
iostructian and learning through mastery as well as relat5?^» 'i)rocedures, the 
effectiveness of most :5f these approaches is dependent on the use of feedback 
and corrective procedures. Evaluation plays^a central role in prpviding 
the feedback on the effects of instruct ion as well at? on the effectiveness 
of the corrective procedures. Properly used the evi0?liation is looked upon t 
both teachers and lelamers as an indispensable tool for ,instructi6n and 
learning, especially when th^ formative evaluation is not used to grade ' 
or judge either teacher or student. 

Many countries have been experimenting with different mastery learning 

* • ■ \. ' , * 

strategies.' Typical^ly, they are finding that -after lI^*-* formative fests and 
corrective procedures are developed by evaluation and curriculum specialists, 
. the cos La of maoLciry learning strategies are negligible. Furthermore, they 
are finding that the outcomes in terms of final achievement, student 
attitudiea toward the learning, and improvement in student general ability 
to learn under school conditions are so great as,, to repr**sent positive 
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- ' human development in its own right aa well as economic b*?nefits which 
are far greater than aight be expected from the time or other costs 
incurred • 

However, for the purposes of this paper, the main joint to be stress«r^d 
is that the use of evaluation as an integral part. of inatruction and learning 
has enormous consequences. We must continue to search I'or additional effective 
ways in which evaluation can contribute to the teaching-learning process- 
as an integral part of that process 

A. Evaluation to determine the effectiveness of Instruction and learning 

Much of the use of evaluation has been to determine the learning out- 
„comes of particular types of curriculum and instruction. Typically, the 
attempt has been made- to construct evaluation procedures that are 
appropriate to a particular educational program, curriculum, or instructional 
approach. Then, an appropriate research design and sampling procedure 
has been chosen to determine whether in fact the educational program, 
course, etc. did have specific traceable effects on the student learning. 

The use of evaluation procedures in the development and appraisal of 
a new curriculum has already been ably presented by Dr. Lewy so I will say, 
little more on this. c 

Rarely d^^es a nation restrict itself to single educational program, 
curriculum, or instructional approach for all students of a particular 
age or grade. Evaluation is useful in determining the relative effective- 
ness of the different approaches to instruction and learning within a nation. 



LValuation used this way has characteristically been used to determine whether 



alternative A is, in terms of student learning, more effective than 
alternatives B, C, D, etc. (alternatives may be programs, courses, curricula, 
teaching methods, class si^^^nstructional strategies, etc . In most of 
the research using this evaluation approach it has characteristically been 
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found that the "opportunity to learn" particular content and Objectives in 
a particular alternative is highly related to the evaluation results for 
that alternative. That is, if students are taught x, y, and z they tend 
to learn x, y, and z, while if they are taught^ only x and z they learn 
accordingly. This seems so obvious that one wonders why evaluation is 
necessary. 

' However, there are great discrepancies between what an educational 
program is intended to accomplish, what students are actually given an 
opportunity to learn , and what students actually learn — and the discrepancie 
have to do with what happeyis in particular classrooms (opportunity to learn) 
in relation to what was intended and the evaluation results. 

Thus , , the\.basic problem of the effects of an educational alternative 
is dependent on the linkages between the intended effects of an alternative 
(course, program, etc.), what happens in the school or classroom, and the 
evaluation results. If an educational program is designed to produce a 
particular set of results, we must insure that the appropriate use of the 
program actually takes place in the classroom, before we can be certain 
that we are really evaluating the effectiveness of the program or 
alternative. In response to this problem, educational evaluators and 
researchers now seek to establish what actually takes place in the classroom s 
they evaluate before claiming the;^ are evaluating the ef fectivjeness -of the 
program. Increasingly, evaluators are selecting classrooms and teachers 
where they are certain that the program is actually being implemented In 
.the intended ways before applying their evaluation procedures . 

Once th^y can satisfy themselves that the classrooms or teachers are 
fully implementing' the intended curriculum, program, or method' they can 
determine its effectiveness as well as its dif ficuities . Then, the research 
moves to the problems of how the program can be fully implemented in other 
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classrooms — training of teachers, orientation of students, appropriate 
materials, or the supporting conditions of morale, educational leadership, 
supervisors, consultants, etc. which are necessary for its full implementation 
with other teachers and classrooms. 

P erhaps the u ial i i lebbuir to be l eaine"^' from the attempts to evaluate 

new early childhood educational programs (Headstart), new programs for the 
disadvantaged students (compensatory education), programmed instruction, 

r 

new curricula (new mathematics, new biolo^-^y, physics, chemistry, etc.) and 
.new teaching strategies is that there are great gaps between the intended 
new program and its full realization in the classrooms. In fact, one has 
>to search very carefully before finding the few classrooms whfere t« iw 
program Is fully realized. Policy makers who sponsor and give economic 
support to the new educational' alternatives must be aware that good 
intentions (especially new ones) are not enough in education. The problems 
of how the good intentions can be fully implemented in the classroom 
must be solved before the new program can be evaluated. 

Closely related to the foregoing points is the increasing use of 
evaluation to determine how an alternative can be modified and improved. 
New approaches to education are rarely pTerfect and seldom are they universal 
panaceas. At one time evaluation was used to determine whether alternative 
A was better than B, or C. It mastered little that the statistical 
significance of the difference between the alternatives was rarely matched 
by \e educational or social significance of the differences. 

^ Evaluation ^ increasingly Is used to determine not only which 
alternatives are superior, but also in what respects can they be further 
improved. A new curriculum or program may be excellent in terms of 
certain characteristics but should be .modified in tertns of other character- 
istics. When tjie evaluation and, other data are propej^ly^ analyzed they. 

16 
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reveal what is excellent about an educational alternative, what is good b^*" 
coula be improved, and what is poor and needs much further work. 

For the educational policy maker or administrator, the basic point i^ 
that raajor changes in programs should be instituted only when there is 
clear evidence that a particular existing program is poor in all respects* 
Improvements and modifications in existing programs may be more effective 
than the creation of entirely new programs. Not only may they be improve^ 
on the basis of the evaluation evidence — it is likely that the enormous 
costs (economic as well as human) of introducing great changes in teacher^' 
tna^terials , and educational points of view will be effective only when all 
aspects of the new program are working effectively in the classrooms and 
school. Smaller changes cause less dislocation in the schools and may, 
under appropriate conditions, be more effective in promoting improved 
instruction and student learning than will completely new programs.- 

A final point to be made on evaluation and effectiveness of instructs- 
and learning is that times and conditions change. An educational program 
that is very effective at one time may in a number of years be less 
effective. A new curriculum which works superbly in year X may in year X-^^ 
work Very . poorly . The deterioration of particular new programs, curricul^^ 
teaching methods has been well documented, especially in relation to some 
of. the major educational changes introduced during the past decade in the 
U.S. as well as other countries. 

Increasingly, educational evaluation is seen as a quality control 
measure. That is, carefully selected samples of students, classrooms, and 
schools are surveyed at particular times to determine whether a new progr^^ 
that worked well at one time still continues to work well. Or, to determi^^^ 
whether particular aspects of the program need to be modified at particul^^ 
points in time if the program- is to continue to work well in the classroo^^* 
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Sometimes, it is found that the program continues to be effective with 
some students and some teachers or schools, but to work less well in certain 
respects with other students and teachers. Again, the point is that the 
educational policy maker or administrator must not expect that education 
can be a fixed and static thing. Times and conditions change and evaluation 
can reveal when and where the changes require modification and improvements 
in the educational programs. 

, Educational evaluation and education ' 

Edjucation in Western societies is frequently equated with schooling. 
We support schools- to give our children and youth an education. We empower 
schools to give formal recognition to. the amount and type of educatibn an 
individual has compacted by the use of credits, certificates, and academic 
degrees. Most of our writing and research on education deal only Wa "h 
schools and schooling, ' ■ 

u Recently, this equation of educatiti and schooling has been attacked 
by scholars *^of education as well as by more radical reformers who insist 
that much learning can and does take place outside the school. But equally 
important, research on education and research on various aspects of thiR 
society have questioned some of the relatioTxS between the school system and 
other subsystems in the society. 

Research into "the relation bfetweeri the schools and the home environment 
ha,s been one of the more fruitful areas of study stimulated by these questions 
Home is a powerful educational environment, especially during the preschool 
and primary school years. Studies of home environments in the United States, 
as well as in several other countries, reveal the effect of the home on 

language development^ ability to -learn from'adults, attitudes toward school 

• 

learning, and aspirations jfor further' education and the occupational careers 
and life styles associated with education^ It is .clear that when the home 
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^nd the school have congruent learning emphases, the "child has little difficulty 
in his later schooling. But when the home and the school have very divergent 
approaches to life and to learning, the child is likely to l/e penalized 
severely by the school — especially when school attendance is Required for 
ten or more years. 

During the past half-decade we have begun to recognize some of the 
problems raised by disparities between home and school. One approach has 
been to preempt some of the years preceding regular school by placing children 
in preschool programs. Other attempts have been made to alter some aspects, 
of the primary school. Still other efforts have been made to alter th6 
home envir^mient. There is no doubt that these attempts to alter the 
.relations between home and school have raised many problems. The resolution 
of these -problems and the Appropriate relations between home and school will 
concern us for many years -to come. 

Schools and peer groups are increasingly in conflict, and the individual 
appears to learn very different things in these two subsystems of society. 
Especially during adolescence do we find these two subsystems diverging. 
The conflicts between the values emphasized by schools and colleges and the 
values emphasized by various peer groups raise serious questions about the 
ways in which these two sets of values can te more effectively related. 
What we desperately need are research and scholarship which will point the 
way to the resolution of some of the more disturbing conflicts between the 

schools and adolescent peer groups, ' 

f 

Recent research by economists attempts to understand the relationships 
between the economic system of a nation and its educational system. It is 
evident that thn relations between education and economics may be very 
different for societies at different stages of industrialization as well as 
for societies which have very different political systems. The view, that 

19 ■ ■ ■ 



education can be conceived of as investment in human capital has stimulated 

f 

. educators as well as economists to study the economic effects of different 
approaches to education. The view of education as both a consumer or 

T 

cultural good and an investment in huiran capital alters many of our 
traditional views about education and its effects. This area of research 

m 

raises long-terra problems about the consequences of this view for support 
of the schools and support of students in the schools. 

There are other subsystems in a nation — religion, mass media, the 
political system, the status system — which have very complex relations 
with education. Perhaps the main point is that education is not confined 
to the school system and that -very complex educational and other relations 
are found between the schools as a subsystem and the other subsystems' 
within a'^ society. While we have tended to think of a system of schooling 
as relatively insulated. from other parts of the society, it is likely that 
the schools will be under pressure to relate more clearly to the other parts 
of the social system. Undoubtedly, we will come to regard eaucation during 
the school-attenfding period, as well as before and after this period, as 
most appropriately the concern of many aspects of the society. Increasingly 

.7 ' ^ 

we will try to determine what can best be learned in the schools, what can 
best be learned elsewhere, and what can be learned only through an effective 
interrelation of different parts of the social system. 

Evaluation raethads are' gradually being developed to appraise the 
learning of a population both in the school as .well as outside of the 
s.chool. The new ideas on national assessment which are being developed 
in a number of countries are efforts to determine what has been learned 
in the schoo-ls, what has been learned elsewhere , and what has been learned 
in the interaction between the schools and other "subsy3tems in a nation. 
This work is of recent development and it will be sorae time before 
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evaluators are effective in determining both the extent of the learning 
as well as the source. 

Once' again, the point is that education and educational policy makers 
must learn to use evaluation and evaluation data to secure a broader 
picture of the educational resources of a nation than may be secured from 
viewing the schools as the single educational resource. This is p'tobably 
the most- complex problem that educational evaluators and policy makers 
must face. The challenges faced by these broader issues suggest that 
international seminars and conferences may be necessary if the problems 
and progress of various national attempts in this field are to be studied 
and utilized where relevant h> other national groups. 

f^. Implications 

Educational evaluation may corttribute tc the improvement pf education 
in many countries of the world. The enormous resources being expended in 
each country for education makes it mandatory that some forms of educational 
evaluation be used for appraising the effectiveness of particular aspects 
> ^ of a national educational program, for determining where it is in need 

of modification or major changes, and for determining how to maintain and 
/ even improve the effectiveness of the schools as well as the related 
educational resources of the nation. 

The appropriate training of a cadre of highly competent educational 
evaluation spe'Cialists. is a minimum requirement if a nation is to make 
effective use of this rafpidly developing technology. The support of and 
l^he appropriate relations between such specialists, educational policy 

makers, and the educational institutions of the nation is necessary to 

■ \ 

maintain educational evaluation at a high level and to insure that the 

\ , ■ 

' evaluation .methods and results play their appropriate role in the 

continued maintenance and improvement of a complex^educational system. 
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